Model Fitting מבוסס על שיעור שנבנה ע"י טל הסנר

Transcription

1 Model Fttng מבוסס על שיעור שנבנה ע"י טל הסנר

2 מקורות מפוזר על פני ספר הלימוד...

3 Fttng: Motvaton We ve learned how to detect edges, corners, blobs. Now what? We would lke to form a hgher-level, more compact representaton of the features n the mage by groupng multple features accordng to a smple model 9300 Harrs Corners Pkwy, Charlotte, NC

4 Fttng Choose a parametrc model to represent a set of features smple model: lnes smple model: crcles complcated model: car Source: K. Grauman

5 Fttng Choose a parametrc model to represent a set of features Lne, ellpse, splne, etc. Three man questons: What model represents ths set of features best? Whch of several model nstances gets whch feature? How many model nstances are there? Computatonal complexty s mportant It s nfeasble to examne every possble set of parameters and every possble combnaton of features

6 Fttng: Issues Case study: Lne detecton Nose n the measured feature locatons Extraneous data: clutter (outlers), multple lnes Mssng data: occlusons

7 Fttng: Issues If we know whch ponts belong to the lne, how do we fnd the optmal lne parameters? Least squares What f there are outlers? RANSAC What f there are many lnes? Votng methods: Hough transform What f we re not even sure t s a lne? Model selecton

8 התאמת קו לנקודות "תחת-רעש"

9 Least squares lne fttng Data: (x 1, y 1 ),, (x n, y n ) Lne equaton: y = m x + b Fnd (m, b) to mnmze E n 1 ( y mx b) 2 (x, y ) y=mx+b

10 Least squares lne fttng Data: (x 1, y 1 ),, (x n, y n ) Lne equaton: y = m x + b Fnd (m, b) to mnmze Y X XB X db de T T ) ( ) ( ) 2( ) ( ) ( XB XB Y XB Y Y XB Y XB Y XB Y b m x x y y b m x y E T T T T n n n Normal equatons: least squares soluton to XB=Y n b mx y E 1 2 ) ( (x, y ) y=mx+b Y X XB X T T

11 ב- MATLAB : הפתרון למערכת המשוואות XB=Y B = X\Y;

12 Problem wth vertcal least squares Not rotaton-nvarant Fals completely for vertcal lnes

13 Total least squares Dstance between pont (x, y ) and lne ax+by=d (a 2 +b 2 =1): n ax + by d E ax by 1 2 ( (x, yd ) ax+by=d Unt normal: N=(a, b)

14 Total least squares Dstance between pont (x, y ) and lne ax+by=d (a 2 +b 2 =1): ax + by d E n 1 2 ( ax(x by, y d ) ax+by=d Unt normal: N=(a, b) Proof: (from wkpeda)

15 Total least squares Dstance between pont (x, y ) and lne ax+by=d (a 2 +b 2 =1): ax + by d Fnd (a, b, d) to mnmze the sum of squared perpendcular dstances n E ax by E n 1 ( ax by d) 2 2 ( (x d 1, y ) ax+by=d Unt normal: N=(a, b)

16 Total least squares Dstance between pont (x, y ) and lne ax+by=d (a 2 +b 2 =1): ax + by d Fnd (a, b, d) to mnmze the sum of squared perpendcular dstances n d by ax E 1 2 ) ( (x, y ) ax+by=d n d by ax E 1 2 ) ( Unt normal: N=(a, b) 0 ) 2( 1 n d by ax d E 1 1 n n a b d x y ax by n n ) ( ) ( )) ( ) ( ( UN UN b a y y x x y y x x y y b x x a E T n n n 0 ) ( 2 N U U dn de T Soluton to (U T U)N = 0, subject to N 2 = 1: egenvector of U T U assocated wth the smallest egenvalue (least squares soluton to homogeneous lnear system UN = 0)

17 Total least squares y y x x y y x x U n n 1 1 n n n n T y y y y x x y y x x x x U U ) ( ) )( ( ) )( ( ) ( second moment matrx

18 Total least squares y y x x y y x x U n n 1 1 n n n n T y y y y x x y y x x x x U U ) ( ) )( ( ) )( ( ) ( ), ( y x N = (a, b) second moment matrx ), ( y y x x

19 Least squares: Robustness to nose Least squares ft to the red ponts:

20 Least squares: Robustness to nose Least squares ft wth an outler: Problem: squared error heavly penalzes outlers

21 מה קורה כשיש נקודות חיצוניות?

22 RANSAC Random sample consensus (RANSAC): Very general framework for model fttng n the presence of outlers M. A. Fschler, R. C. Bolles. Random Sample Consensus: A Paradgm for Model Fttng wth Applcatons to Image Analyss and Automated Cartography. Comm. of the ACM, Vol 24, pp , 1981.

23 Fttng a Lne Least squares ft

24 RANSAC Select sample of m ponts at random

25 RANSAC Select sample of m ponts at random Calculate model parameters that ft the data n the sample

26 RANSAC Select sample of m ponts at random Calculate model parameters that ft the data n the sample Calculate error functon for each data pont

27 RANSAC Select sample of m ponts at random Calculate model parameters that ft the data n the sample Calculate error functon for each data pont Select data that support current hypothess

28 RANSAC Select sample of m ponts at random Calculate model parameters that ft the data n the sample Calculate error functon for each data pont Select data that support current hypothess

29 RANSAC for lne fttng Repeat N tmes: Draw s ponts unformly at random Ft lne to these s ponts Fnd nlers to ths lne among the remanng ponts (.e., ponts whose dstance from the lne s less than t) If there are d or more nlers, accept the lne and reft usng all nlers

30 Choosng the parameters Intal number of ponts s Typcally mnmum number needed to ft the model Dstance threshold t Choose t so probablty for nler s p (e.g. 0.95) Zero-mean Gaussan nose wth std. dev. σ: t 2 =3.84σ 2 Number of teratons N Choose N so that, wth probablty p, at least one random sample s free from outlers (e.g. p=0.99) (outler rato: e) proporton of outlers e s 5% 10% 20% 25% 30% 40% 50% Source: M. Pollefeys

31 Choosng the parameters Intal number of ponts s Typcally mnmum number needed to ft the model Dstance threshold t Choose t so probablty for nler s p (e.g. 0.95) Zero-mean Gaussan nose wth std. dev. σ: t 2 =3.84σ 2 Number of teratons N Choose N so that, wth probablty p, at least one random sample s free from outlers (e.g. p=0.99) (outler rato: e) Consensus set sze d Should match expected nler rato Source: M. Pollefeys

32 RANSAC pros and cons Pros Smple and general Applcable to many dfferent problems Often works well n practce Cons Lots of parameters to tune Can t always get a good ntalzaton of the model based on the mnmum number of samples Sometmes too many teratons are requred Can fal for extremely low nler ratos We can often do better than brute-force samplng

33 מה קורה כשיש יותר מקו אחד?

34 Votng schemes Let each feature vote for all the models that are compatble wth t Hopefully the nose features wll not vote consstently for any sngle model Mssng data doesn t matter as long as there are enough features remanng to agree on a good model

35 Hough transform An early type of votng scheme General outlne: Dscretze parameter space nto bns For each feature pont n the mage, put a vote n every bn n the parameter space that could have generated ths pont Fnd bns that have the most votes Image space Hough parameter space P.V.C. Hough, Machne Analyss of Bubble Chamber Pctures, Proc. Int. Conf. Hgh Energy Accelerators and Instrumentaton, 1959

36 Parameter space representaton A lne n the mage corresponds to a pont n Hough space Image space Hough parameter space Source: S. Setz

37 Parameter space representaton What does a pont (x 0, y 0 ) n the mage space map to n the Hough space? Image space Hough parameter space

38 Parameter space representaton What does a pont (x 0, y 0 ) n the mage space map to n the Hough space? Image space Hough parameter space

39 Parameter space representaton What does a pont (x 0, y 0 ) n the mage space map to n the Hough space? Answer: the solutons of b = x 0 m + y 0 Ths s a lne n Hough space Image space Hough parameter space

40 Parameter space representaton Where s the lne that contans both (x 0, y 0 ) and (x 1, y 1 )? Image space Hough parameter space (x 1, y 1 ) (x 0, y 0 ) b = x 1 m + y 1

41 Parameter space representaton Where s the lne that contans both (x 0, y 0 ) and (x 1, y 1 )? It s the ntersecton of the lnes b = x 0 m + y 0 and b = x 1 m + y 1 Image space Hough parameter space (x 1, y 1 ) (x 0, y 0 ) b = x 1 m + y 1

42 Lne Detecton by Hough Transform Algorthm: y Quantze Parameter Space Create Accumulator Array Set For each mage edge If A ( m, c) 0 m, c ( m, c) les on the lne: Fnd local maxma n ( x, y ( m, c) A( m, c) A( m, c) A( m, c) 1 c x m y ) A( m, c) ncrement: Parameter Space A( m, c) ( m, c) x

43 Parameter space representaton Problems wth the (m,b) space: Unbounded parameter doman Vertcal lnes requre nfnte m Quck soluton?

44 Parameter space representaton Problems wth the (m,b) space: Unbounded parameter doman Vertcal lnes requre nfnte m Alternatve: polar representaton xcos y sn Each pont wll add a snusod n the (,) parameter space

45 Algorthm outlne Intalze accumulator H to all zeros For each edge pont (x,y) ρ n the mage For θ = 0 to 180 ρ = x cos θ + y sn θ H(θ, ρ) = H(θ, ρ) + 1 θ end end Fnd the value(s) of (θ, ρ) where H(θ, ρ) s a local maxmum The detected lne n the mage s gven by ρ = x cos θ + y sn θ

46 Basc llustraton features votes

47 Other shapes

48 Other shapes Square Crcle

49 Several lnes

50 A more complcated mage

51 ב- MATLAB [H,T,R] = hough(bw); % The followng functon fnds no more than 5 peaks n the Hough matrx H P = houghpeaks(h,5); % Extracts the lne segments (not a part of the algorthm descrbed, but useful) lnes = houghlnes(bw,t,r,p); % Each lne can then be plotted by: for =1:numel(lnes) xy = [lnes().pont1; lnes().pont2]; plot(xy(:,1),xy(:,2)); end

52 Effect of nose features votes

53 Effect of nose features Peak gets fuzzy and hard to locate votes

54 Effect of nose Number of votes for a lne of 20 ponts wth ncreasng nose:

55 Random ponts features votes Unform nose can lead to spurous peaks n the array

56 Random ponts As the level of unform nose ncreases, the maxmum number of votes ncreases too:

57 Dealng wth nose Choose a good grd / dscretzaton Too coarse: large votes obtaned when too many dfferent lnes correspond to a sngle bucket Too fne: mss lnes because some ponts that are not exactly collnear cast votes for dfferent buckets Increment neghborng bns (smoothng n accumulator array) Try to get rd of rrelevant features Take only edge ponts wth sgnfcant gradent magntude

58 Incorporatng mage gradents Recall: when we detect an edge pont, we also know ts gradent drecton But ths means that the lne s unquely determned! Modfed Hough transform: For each edge pont (x,y) θ = gradent orentaton at (x,y) ρ = x cos θ + y sn θ H(θ, ρ) = H(θ, ρ) + 1 end

59 Fndng Crcles by Hough Transform Equaton of Crcle: ( x a) ( y b) r If radus s known: (2D Hough Space) Accumulator Array A( a, b)

60 אם הרדיוס R ידוע...

61 Fndng Crcles by Hough Transform Equaton of Crcle: ( x a) ( y b) r If radus s not known: 3D Hough Space! Use Accumulator array A( a, b, r) What s the surface n the hough space?

62 אם הרדיוס לא ידוע... a, b, R r מרחב הפרמטרים גדל: b a

63 Usng Gradent Informaton Gradent nformaton can save lot of computaton: Edge Locaton Edge Drecton ( x, y ) Assume radus s known: a x r cos b y r sn Need to ncrement only one pont n Accumulator!!

64 מעבר לצורות "פשוטות"

65 Generalzed Hough transform We want to fnd a shape defned by ts boundary ponts and a reference pont a D. Ballard, Generalzng the Hough Transform to Detect Arbtrary Shapes, Pattern Recognton 13(2), 1981, pp

66 Generalzed Hough Transform Model Shape NOT descrbed by equaton

67 Generalzed Hough Transform Model Shape NOT descrbed by equaton

68 Generalzed Hough Transform Fnd Object Center ( c c x, y ) gven edges x,, ) ( y Create Accumulator Array Intalze: For each edge pont For each entry Increment Accumulator: Fnd Local Maxma n A( x c, yc) A( xc, yc) 0 ( xc, yc) ( x, y, ) r k x y c c A( x c, yc) n table, compute: x y r A( x, y ) A( x, y ) 1 c k r k cos sn c k k c c

69 Generalzed Hough transform Assumpton: translaton s the only transformaton here,.e., orentaton and scale are fxed. How can we generalze the dea to a model that can be rotated or scaled? Source: K. Grauman

70

71 Hough transform: Dscusson Pros Can deal wth non-localty and occluson Can detect multple nstances of a model Some robustness to nose: nose ponts unlkely to contrbute consstently to any sngle bn Cons Complexty of search tme ncreases exponentally wth the number of model parameters Non-target shapes can produce spurous peaks n parameter space It s hard to pck a good grd sze Hough transform vs. RANSAC

72 התאמת נקודות לקבוצות

73 התאמת נקודות לקבוצות Clusterng Segmentaton K-means

74 מוטיבציה נרצה לחלק את התמונה לאזורים לפי צבע

75

76

77

78

79

80

81

82

83 תוצאות K=10 K=5 מקור

84 K-means clusterng Want to mnmze sum of squared Eucldean dstances between ponts x and ther nearest cluster centers m k Algorthm: D( X, M ) Randomly ntalze K cluster centers Iterate untl convergence: ( Assgn each data pont to the nearest center cluster k cluster pont n k x m k Recompute each cluster center as the mean of all ponts assgned to t 2 )

85 PUTTING IT ALL TOGETHER דוגמה: מערכת לאיתור מכוניות

86 Implct shape models: Tranng 1. Buld codebook of patches around extracted nterest ponts usng k-means clusterng B. Lebe, A. Leonards, and B. Schele, Combned Object Categorzaton and Segmentaton wth an Implct Shape Model, ECCV Workshop on Statstcal Learnng n Computer Vson 2004

87 Implct shape models: Tranng 1. Buld codebook of patches around extracted nterest ponts usng k-means clusterng 2. Map the patch around each nterest pont to closest codebook entry

88 Implct shape models: Tranng 1. Buld codebook of patches around extracted nterest ponts usng k-means clusterng 2. Map the patch around each nterest pont to closest codebook entry 3. For each codebook entry, store all postons t was found, relatve to object center

89 Implct shape models: Testng 1. Gven test mage, extract patches, match to codebook entry 2. Cast votes for possble postons of object center 3. Search for maxma n votng space 4. Extract weghted segmentaton mask based on stored masks for the codebook occurrences

90 אז מה ראינו היום?

91 סיכום התאמת נקודות לישר ריבועים פחותים שתי גרסאות RANSAC האאף האאף למעגלים ולצורות כלליות קיבוץ נקודות לקבוצות k-means clusterng דוגמה לאלגוריתם לאיתור מכוניות

92 מקורות לשקפים מלבד אלו שצוינו במפורש, שקפים מבוססים על אלו ש: Svetlana Lazebnk Ondřej Chum Jason Lawrence Szymon Rusnkewcz Harvey Rhody Utkarsh Snha